
<!DOCTYPE html>

<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

    <title>Test Action Reference &#8212; LAVA 2024.05 documentation</title>
    <link rel="stylesheet" type="text/css" href="_static/pygments.css" />
    <link rel="stylesheet" type="text/css" href="_static/bootstrap-sphinx.css" />
    <script data-url_root="./" id="documentation_options" src="_static/documentation_options.js"></script>
    <script src="_static/jquery.js"></script>
    <script src="_static/underscore.js"></script>
    <script src="_static/_sphinx_javascript_frameworks_compat.js"></script>
    <script src="_static/doctools.js"></script>
    <script src="_static/sphinx_highlight.js"></script>
    <link rel="shortcut icon" href="_static/favicon.ico"/>
    <link rel="index" title="Index" href="genindex.html" />
    <link rel="search" title="Search" href="search.html" />
    <link rel="next" title="Command Action Reference" href="actions-command.html" />
    <link rel="prev" title="Boot Action Reference" href="actions-boot.html" />
    <link rel="canonical" href="https://docs.lavasoftware.org/lava/actions-test.html" />
  
<meta charset='utf-8'>
<meta http-equiv='X-UA-Compatible' content='IE=edge,chrome=1'>
<meta name='viewport' content='width=device-width, initial-scale=1.0, maximum-scale=1'>
<meta name="apple-mobile-web-app-capable" content="yes">
<script type="text/javascript" src="_static/js/jquery-1.12.4.min.js"></script>
<script type="text/javascript" src="_static/js/jquery-fix.js"></script>
<script type="text/javascript" src="_static/bootstrap-3.4.1/js/bootstrap.min.js"></script>
<script type="text/javascript" src="_static/bootstrap-sphinx.js"></script>


  </head><body>

  <div id="navbar" class="navbar navbar-default navbar-fixed-top">
    <div class="container">
      <div class="navbar-header">
        <!-- .btn-navbar is used as the toggle for collapsed navbar content -->
        <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".nav-collapse">
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
          <span class="icon-bar"></span>
        </button>
        <a class="navbar-brand" href="index.html"><span><img src="_static/lava.png"></span>
          LAVA</a>
        <span class="navbar-text navbar-version pull-left"><b>2024.05</b></span>
      </div>

        <div class="collapse navbar-collapse nav-collapse">
          <ul class="nav navbar-nav">
            
                <li><a href="genindex.html">Index</a></li>
                <li><a href="contents.html">Contents</a></li>
            
            
              <li class="dropdown globaltoc-container">
  <a role="button"
     id="dLabelGlobalToc"
     data-toggle="dropdown"
     data-target="#"
     href="index.html">Site <b class="caret"></b></a>
  <ul class="dropdown-menu globaltoc"
      role="menu"
      aria-labelledby="dLabelGlobalToc"><ul class="current">
<li class="toctree-l1"><a class="reference internal" href="index.html">Introduction to LAVA</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="contents.html">Contents</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="glossary.html">Glossary of terms</a></li>
</ul>
<ul>
<li class="toctree-l1"><a class="reference internal" href="support.html">Getting support</a></li>
</ul>
</ul>
</li>
              
                <li class="dropdown">
  <a role="button"
     id="dLabelLocalToc"
     data-toggle="dropdown"
     data-target="#"
     href="#">Page <b class="caret"></b></a>
  <ul class="dropdown-menu localtoc"
      role="menu"
      aria-labelledby="dLabelLocalToc"><ul>
<li><a class="reference internal" href="#">Test Action Reference</a><ul>
<li><a class="reference internal" href="#definitions">Definitions</a><ul>
<li><a class="reference internal" href="#repository">repository</a></li>
<li><a class="reference internal" href="#from">from</a><ul>
<li><a class="reference internal" href="#git">git</a></li>
<li><a class="reference internal" href="#inline">inline</a></li>
<li><a class="reference internal" href="#path">path</a></li>
<li><a class="reference internal" href="#name">name</a></li>
<li><a class="reference internal" href="#params">params</a></li>
</ul>
</li>
<li><a class="reference internal" href="#skipping-elements-of-test-definitions">Skipping elements of test definitions</a><ul>
<li><a class="reference internal" href="#deps">deps</a></li>
<li><a class="reference internal" href="#keys">keys</a></li>
<li><a class="reference internal" href="#sources">sources</a></li>
<li><a class="reference internal" href="#steps">steps</a></li>
<li><a class="reference internal" href="#all">all</a></li>
</ul>
</li>
<li><a class="reference internal" href="#inline-test-definition-example">Inline test definition example</a></li>
<li><a class="reference internal" href="#additional-support">Additional support</a><ul>
<li><a class="reference internal" href="#result-checks">Result checks</a></li>
<li><a class="reference internal" href="#testsets">TestSets</a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#interactive">Interactive</a><ul>
<li><a class="reference internal" href="#id2">name</a></li>
<li><a class="reference internal" href="#prompts">prompts</a></li>
<li><a class="reference internal" href="#echo">echo</a></li>
<li><a class="reference internal" href="#script">script</a></li>
</ul>
</li>
<li><a class="reference internal" href="#monitors">Monitors</a><ul>
<li><a class="reference internal" href="#id3">name</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</ul>
</li>
              
            
            
              
                
  <li>
    <a href="actions-boot.html" title="Previous Chapter: Boot Action Reference"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">&laquo; Boot Action Reference</span>
    </a>
  </li>
  <li>
    <a href="actions-command.html" title="Next Chapter: Command Action Reference"><span class="glyphicon glyphicon-chevron-right visible-sm"></span><span class="hidden-sm hidden-tablet">Command Actio... &raquo;</span>
    </a>
  </li>
              
            
            
            
            
              <li class="hidden-sm"></li>
            
          </ul>

          
            
<form class="navbar-form navbar-right" action="search.html" method="get">
 <div class="form-group">
  <input type="text" name="q" class="form-control" placeholder="Search" />
 </div>
  <input type="hidden" name="check_keywords" value="yes" />
  <input type="hidden" name="area" value="default" />
</form>
          
        </div>
    </div>
  </div>

<div class="container">
  <div class="row">
    <div class="body col-md-12 content" role="main">
      
  <section id="test-action-reference">
<span id="test-action"></span><h1><a class="toc-backref" href="#contents" role="doc-backlink">Test Action Reference</a><a class="headerlink" href="#test-action-reference" title="Permalink to this heading">¶</a></h1>
<p>The pipeline jobs (LAVA v2) have retained compatibility with respect to the
content of Lava-Test-Shell Test Definitions although the submission format
has changed:</p>
<ol class="arabic simple">
<li><p>The <a class="reference internal" href="#test-action"><span class="std std-ref">test action</span></a> will <strong>never</strong> boot the device -
a <a class="reference internal" href="actions-boot.html#boot-action"><span class="std std-ref">boot action</span></a> <strong>must</strong> be specified. Multiple test
operations need to be specified as multiple definitions listed within
the same test block.</p></li>
<li><p>The LAVA support scripts are prepared by the <a class="reference internal" href="actions-deploy.html#deploy-action"><span class="std std-ref">deploy action</span></a>
action and the same scripts will be used for all test definitions
until next <code class="docutils literal notranslate"><span class="pre">deploy</span></code> block is encountered.</p></li>
</ol>
<p>There are 3 types of test actions:</p>
<ul class="simple">
<li><p><a class="reference internal" href="#test-action-definitions"><span class="std std-ref">lava-test-shell definitions</span></a>
(YAML directive: <code class="docutils literal notranslate"><span class="pre">definitions</span></code>) are used for POSIX compliant operating
systems on the <a class="reference internal" href="glossary.html#term-DUT"><span class="xref std std-term">DUT</span></a>. The deployed system is expected to support
a POSIX shell environment (<code class="docutils literal notranslate"><span class="pre">/bin/ash</span></code>, <code class="docutils literal notranslate"><span class="pre">/bin/dash</span></code> or <code class="docutils literal notranslate"><span class="pre">/bin/bash</span></code>
are the most common) so that LAVA can execute the LAVA Test Shell Helper
scripts.</p></li>
<li><p><a class="reference internal" href="#monitor-test-action"><span class="std std-ref">Output monitors</span></a> (YAML directive:
<code class="docutils literal notranslate"><span class="pre">monitors</span></code>) are used for devices which have no POSIX shell and start
the test (and corresponding output) immediately after booting, for
example microcontroller/IoT boards.</p></li>
<li><p><a class="reference internal" href="#interactive-test-action"><span class="std std-ref">Interactive tests</span></a> (YAML directive:
<code class="docutils literal notranslate"><span class="pre">interactive</span></code>) are further extension of “monitor” tests idea, allowing
not just matching some output from a device, but also feeding some input.
They are useful for non-POSIX shells like bootloaders (u-boot for instance)
and other interactive command-line applications.</p></li>
</ul>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="writing-tests.html#lava-test-helpers"><span class="std std-ref">LAVA Test Helpers</span></a></p>
</div>
<nav class="contents" id="contents">
<p class="topic-title">Contents</p>
<ul class="simple">
<li><p><a class="reference internal" href="#test-action-reference" id="id4">Test Action Reference</a></p>
<ul>
<li><p><a class="reference internal" href="#definitions" id="id5">Definitions</a></p>
<ul>
<li><p><a class="reference internal" href="#repository" id="id6">repository</a></p></li>
<li><p><a class="reference internal" href="#from" id="id7">from</a></p>
<ul>
<li><p><a class="reference internal" href="#git" id="id8">git</a></p></li>
<li><p><a class="reference internal" href="#inline" id="id9">inline</a></p></li>
<li><p><a class="reference internal" href="#path" id="id10">path</a></p></li>
<li><p><a class="reference internal" href="#name" id="id11">name</a></p></li>
<li><p><a class="reference internal" href="#params" id="id12">params</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#skipping-elements-of-test-definitions" id="id13">Skipping elements of test definitions</a></p>
<ul>
<li><p><a class="reference internal" href="#deps" id="id14">deps</a></p></li>
<li><p><a class="reference internal" href="#keys" id="id15">keys</a></p></li>
<li><p><a class="reference internal" href="#sources" id="id16">sources</a></p></li>
<li><p><a class="reference internal" href="#steps" id="id17">steps</a></p></li>
<li><p><a class="reference internal" href="#all" id="id18">all</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#inline-test-definition-example" id="id19">Inline test definition example</a></p></li>
<li><p><a class="reference internal" href="#additional-support" id="id20">Additional support</a></p>
<ul>
<li><p><a class="reference internal" href="#result-checks" id="id21">Result checks</a></p></li>
<li><p><a class="reference internal" href="#testsets" id="id22">TestSets</a></p></li>
</ul>
</li>
</ul>
</li>
<li><p><a class="reference internal" href="#interactive" id="id23">Interactive</a></p>
<ul>
<li><p><a class="reference internal" href="#id2" id="id24">name</a></p></li>
<li><p><a class="reference internal" href="#prompts" id="id25">prompts</a></p></li>
<li><p><a class="reference internal" href="#echo" id="id26">echo</a></p></li>
<li><p><a class="reference internal" href="#script" id="id27">script</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#monitors" id="id28">Monitors</a></p>
<ul>
<li><p><a class="reference internal" href="#id3" id="id29">name</a></p></li>
</ul>
</li>
</ul>
</li>
</ul>
</nav>
<section id="definitions">
<span id="test-action-definitions"></span><span id="index-0"></span><h2><a class="toc-backref" href="#contents" role="doc-backlink">Definitions</a><a class="headerlink" href="#definitions" title="Permalink to this heading">¶</a></h2>
<section id="repository">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">repository</a><a class="headerlink" href="#repository" title="Permalink to this heading">¶</a></h3>
<p>A publicly readable repository location.</p>
</section>
<section id="from">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">from</a><a class="headerlink" href="#from" title="Permalink to this heading">¶</a></h3>
<p>The type of the repository is <strong>not</strong> guessed, it <strong>must</strong> be specified
explicitly. Support is available for <code class="docutils literal notranslate"><span class="pre">git</span></code>. Support is planned
for <code class="docutils literal notranslate"><span class="pre">url</span></code> and <code class="docutils literal notranslate"><span class="pre">tar</span></code>.</p>
<section id="git">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">git</a><a class="headerlink" href="#git" title="Permalink to this heading">¶</a></h4>
<p>A remote git repository which needs to be cloned by the dispatcher.</p>
</section>
<section id="inline">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">inline</a><a class="headerlink" href="#inline" title="Permalink to this heading">¶</a></h4>
<p>A simple test definition present in the same file as the job submission,
instead of from a separate file or VCS repository. This allows tests to be run
based on a single file. When combined with <code class="docutils literal notranslate"><span class="pre">file://</span></code> URLs to the <code class="docutils literal notranslate"><span class="pre">deploy</span></code>
parameters, this allows tests to run without needing external access. See
<a class="reference internal" href="#inline-test-definition-example"><span class="std std-ref">Inline test definition example</span></a>.</p>
</section>
<section id="path">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">path</a><a class="headerlink" href="#path" title="Permalink to this heading">¶</a></h4>
<p>The path within that repository to the YAML file containing the test
definition.</p>
</section>
<section id="name">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">name</a><a class="headerlink" href="#name" title="Permalink to this heading">¶</a></h4>
<p>(required) - replaces the name from the YAML.</p>
</section>
<section id="params">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">params</a><a class="headerlink" href="#params" title="Permalink to this heading">¶</a></h4>
<p>(optional): Pass parameters to the Lava Test Shell Definition. The format is a
YAML dictionary - the key is the name of the variable to be made available to
the test shell, the value is the value of that variable.</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">    </span><span class="nt">definitions</span><span class="p">:</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">https://gitlab.com/lava/hacking-session.git</span>
<span class="w">      </span><span class="nt">from</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git</span>
<span class="w">      </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">hacking-session-debian.yaml</span>
<span class="w">      </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">hacking</span>
<span class="w">      </span><span class="nt">params</span><span class="p">:</span>
<span class="w">        </span><span class="nt">IRC_USER</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;&quot;</span>
<span class="w">        </span><span class="nt">PUB_KEY</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;&quot;</span>
</pre></div>
</div>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">    </span><span class="nt">definitions</span><span class="p">:</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git://git.linaro.org/lava-team/lava-functional-tests.git</span>
<span class="w">      </span><span class="nt">from</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git</span>
<span class="w">      </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-shell/smoke-tests-basic.yaml</span>
<span class="w">      </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">smoke-tests</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">https://git.linaro.org/lava-team/lava-functional-tests.git</span>
<span class="w">      </span><span class="nt">from</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git</span>
<span class="w">      </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-shell/single-node/singlenode03.yaml</span>
<span class="w">      </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">singlenode-advanced</span>
</pre></div>
</div>
</section>
</section>
<section id="skipping-elements-of-test-definitions">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">Skipping elements of test definitions</a><a class="headerlink" href="#skipping-elements-of-test-definitions" title="Permalink to this heading">¶</a></h3>
<p>When a single test definition is to be used across multiple deployment types
(e.g. Debian and OpenEmbedded), it may become necessary to only perform certain
actions within that definition in specific jobs. The <code class="docutils literal notranslate"><span class="pre">skip_install</span></code> support
has been migrated from V1 for compatibility. Other methods of optimizing test
definitions for specific deployments may be implemented in V2 later.</p>
<p>The available steps which can be (individually) skipped are:</p>
<section id="deps">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">deps</a><a class="headerlink" href="#deps" title="Permalink to this heading">¶</a></h4>
<p>skip running <code class="docutils literal notranslate"><span class="pre">lava-install-packages</span></code> for the <code class="docutils literal notranslate"><span class="pre">deps:</span></code> list of the
<code class="docutils literal notranslate"><span class="pre">install:</span></code> section of the definition.</p>
</section>
<section id="keys">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">keys</a><a class="headerlink" href="#keys" title="Permalink to this heading">¶</a></h4>
<p>skip running <code class="docutils literal notranslate"><span class="pre">lava-add-keys</span></code> for the <code class="docutils literal notranslate"><span class="pre">keys:</span></code> list of the <code class="docutils literal notranslate"><span class="pre">install:</span></code>
section of the definition.</p>
</section>
<section id="sources">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">sources</a><a class="headerlink" href="#sources" title="Permalink to this heading">¶</a></h4>
<p>skip running <code class="docutils literal notranslate"><span class="pre">lava-add-sources</span></code> for the <code class="docutils literal notranslate"><span class="pre">sources:</span></code> list of the <code class="docutils literal notranslate"><span class="pre">install:</span></code>
section of the definition.</p>
</section>
<section id="steps">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">steps</a><a class="headerlink" href="#steps" title="Permalink to this heading">¶</a></h4>
<p>skip running any of the <code class="docutils literal notranslate"><span class="pre">steps:</span></code> of the <code class="docutils literal notranslate"><span class="pre">install:</span></code> section of the
definition.</p>
</section>
<section id="all">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">all</a><a class="headerlink" href="#all" title="Permalink to this heading">¶</a></h4>
<p>identical to <code class="docutils literal notranslate"><span class="pre">['deps',</span> <span class="pre">'keys',</span> <span class="pre">'sources',</span> <span class="pre">'steps']</span></code></p>
<p>Example syntax:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">    </span><span class="nt">failure_retry</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">3</span>
<span class="w">    </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">kvm-basic-singlenode</span>
<span class="w">    </span><span class="nt">timeout</span><span class="p">:</span>
<span class="w">      </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">5</span>
<span class="w">    </span><span class="nt">definitions</span><span class="p">:</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git://git.linaro.org/lava-team/lava-functional-tests.git</span>
<span class="w">      </span><span class="nt">from</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git</span>
<span class="w">      </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-shell/smoke-tests-basic.yaml</span>
<span class="w">      </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">smoke-tests</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">http://git.linaro.org/lava-team/lava-functional-tests.git</span>
<span class="w">      </span><span class="nt">skip_install</span><span class="p">:</span>
<span class="w">      </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">all</span>
<span class="w">      </span><span class="nt">from</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git</span>
<span class="w">      </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-shell/single-node/singlenode03.yaml</span>
<span class="w">      </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">singlenode-advanced</span>
</pre></div>
</div>
<p>The following will skip dependency installation and key addition in
the same definition:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">    </span><span class="nt">failure_retry</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">3</span>
<span class="w">    </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">kvm-basic-singlenode</span>
<span class="w">    </span><span class="nt">timeout</span><span class="p">:</span>
<span class="w">      </span><span class="nt">minutes</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">5</span>
<span class="w">    </span><span class="nt">definitions</span><span class="p">:</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git://git.linaro.org/lava-team/lava-functional-tests.git</span>
<span class="w">      </span><span class="nt">from</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git</span>
<span class="w">      </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-shell/smoke-tests-basic.yaml</span>
<span class="w">      </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">smoke-tests</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">http://git.linaro.org/lava-team/lava-functional-tests.git</span>
<span class="w">      </span><span class="nt">skip_install</span><span class="p">:</span>
<span class="w">      </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">deps</span>
<span class="w">      </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">keys</span>
<span class="w">      </span><span class="nt">from</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">git</span>
<span class="w">      </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-shell/single-node/singlenode03.yaml</span>
<span class="w">      </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">singlenode-advanced</span>
</pre></div>
</div>
</section>
</section>
<section id="inline-test-definition-example">
<span id="id1"></span><h3><a class="toc-backref" href="#contents" role="doc-backlink">Inline test definition example</a><a class="headerlink" href="#inline-test-definition-example" title="Permalink to this heading">¶</a></h3>
<p><a class="reference external" href="https://gitlab.com/lava/lava/-/raw/master/tests/lava_dispatcher/sample_jobs/kvm-inline.yaml">https://gitlab.com/lava/lava/-/raw/master/tests/lava_dispatcher/sample_jobs/kvm-inline.yaml</a></p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">    </span><span class="nt">failure_retry</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">3</span>
<span class="w">    </span><span class="nt">definitions</span><span class="p">:</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span>
<span class="w">        </span><span class="nt">metadata</span><span class="p">:</span>
<span class="w">          </span><span class="nt">format</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Lava-Test Test Definition 1.0</span>
<span class="w">          </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">smoke-tests-basic</span>
<span class="w">          </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;Basic</span><span class="nv"> </span><span class="s">system</span><span class="nv"> </span><span class="s">test</span><span class="nv"> </span><span class="s">command</span><span class="nv"> </span><span class="s">for</span><span class="nv"> </span><span class="s">Linaro</span><span class="nv"> </span><span class="s">Ubuntu</span><span class="nv"> </span><span class="s">images&quot;</span>
<span class="w">          </span><span class="nt">os</span><span class="p">:</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ubuntu</span>
<span class="w">          </span><span class="nt">scope</span><span class="p">:</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">functional</span>
<span class="w">          </span><span class="nt">devices</span><span class="p">:</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">panda</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">panda-es</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">arndale</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">vexpress-a9</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">vexpress-tc2</span>
<span class="w">        </span><span class="nt">run</span><span class="p">:</span>
<span class="w">          </span><span class="nt">steps</span><span class="p">:</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case linux-INLINE-pwd --shell pwd</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case linux-INLINE-uname --shell uname -a</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case linux-INLINE-vmstat --shell vmstat</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case linux-INLINE-ifconfig --shell ifconfig -a</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case linux-INLINE-lscpu --shell lscpu</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case linux-INLINE-lsusb --shell lsusb</span>
<span class="w">          </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case linux-INLINE-lsb_release --shell lsb_release -a</span>
<span class="w">      </span><span class="nt">from</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">inline</span>
<span class="w">      </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">smoke-tests-inline</span>
<span class="w">      </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">inline/smoke-tests-basic.yaml</span>
</pre></div>
</div>
</section>
<section id="additional-support">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">Additional support</a><a class="headerlink" href="#additional-support" title="Permalink to this heading">¶</a></h3>
<p>The V2 dispatcher supports some additional elements in Lava Test Shell which
will not be supported in the older V1 dispatcher.</p>
<section id="result-checks">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">Result checks</a><a class="headerlink" href="#result-checks" title="Permalink to this heading">¶</a></h4>
<p>LAVA collects results from internal operations, these form the <code class="docutils literal notranslate"><span class="pre">lava</span></code> test
suite results as well as from the submitted test definitions. The full set of
results for a job are available at:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>results/1234
</pre></div>
</div>
<p>LAVA records when a submitted test definition starts execution on the test
device. If the number of test definitions which started is not the same as the
number of test definitions submitted (allowing for the <code class="docutils literal notranslate"><span class="pre">lava</span></code> test suite
results), a warning will be displayed on this page.</p>
</section>
<section id="testsets">
<h4><a class="toc-backref" href="#contents" role="doc-backlink">TestSets</a><a class="headerlink" href="#testsets" title="Permalink to this heading">¶</a></h4>
<p>A TestSet is a group of lava test cases which will be collated within the LAVA
Results. This allows queries to look at a set of related test cases within a
single definition.</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">   </span><span class="nt">definitions</span><span class="p">:</span>
<span class="w">   </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">repository</span><span class="p">:</span>
<span class="w">       </span><span class="nt">run</span><span class="p">:</span>
<span class="w">         </span><span class="nt">steps</span><span class="p">:</span>
<span class="w">         </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-set start first_set</span>
<span class="w">         </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case date --shell ntpdate-debian</span>
<span class="w">         </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ls /</span>
<span class="w">         </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case mount --shell mount</span>
<span class="w">         </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-set stop</span>
<span class="w">         </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">lava-test-case uname --shell uname -a</span>
</pre></div>
</div>
<p>This results in the <code class="docutils literal notranslate"><span class="pre">date</span></code> and <code class="docutils literal notranslate"><span class="pre">mount</span></code> test cases being included into a
<code class="docutils literal notranslate"><span class="pre">first_set</span></code> TestSet, independent of other test cases. The TestSet is
concluded with the <code class="docutils literal notranslate"><span class="pre">lava-test-set</span> <span class="pre">stop</span></code> command, meaning that the <code class="docutils literal notranslate"><span class="pre">uname</span></code>
test case has no test set, providing a structure like:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">results</span><span class="p">:</span>
<span class="w">  </span><span class="nt">first_set</span><span class="p">:</span>
<span class="w">    </span><span class="nt">date</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pass</span>
<span class="w">    </span><span class="nt">mount</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pass</span>
<span class="w">  </span><span class="nt">uname</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pass</span>
</pre></div>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="p">{</span><span class="s1">&#39;results&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;first_set&#39;</span><span class="p">:</span> <span class="p">{</span><span class="s1">&#39;date&#39;</span><span class="p">:</span> <span class="s1">&#39;pass&#39;</span><span class="p">,</span> <span class="s1">&#39;mount&#39;</span><span class="p">:</span> <span class="s1">&#39;pass&#39;</span><span class="p">},</span> <span class="s1">&#39;uname&#39;</span><span class="p">:</span> <span class="s1">&#39;pass&#39;</span><span class="p">}}</span>
</pre></div>
</div>
<p>Each TestSet name must be valid as a URL, which is consistent with the
requirements for test definition names and test case names in the V1
dispatcher.</p>
<p>For TestJob <code class="docutils literal notranslate"><span class="pre">1234</span></code>, the <code class="docutils literal notranslate"><span class="pre">uname</span></code> test case would appear as:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>results/1234/testset-def/uname
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">date</span></code> and <code class="docutils literal notranslate"><span class="pre">mount</span></code> test cases are referenced via the TestSet:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>results/1234/testset-def/first_set/date
results/1234/testset-def/first_set/mount
</pre></div>
</div>
<p>A single test definition can start and stop different TestSets in sequence, as
long as the name of each TestSet is unique for that test definition.</p>
</section>
</section>
</section>
<section id="interactive">
<span id="interactive-test-action"></span><span id="index-1"></span><h2><a class="toc-backref" href="#contents" role="doc-backlink">Interactive</a><a class="headerlink" href="#interactive" title="Permalink to this heading">¶</a></h2>
<p>An interactive test action allows to interact with a non-POSIX shell or
just arbitrary interactive application. For instance, the shell of u-boot
bootloader.</p>
<p>The workflow of the interactive test action is:</p>
<ul class="simple">
<li><p>send the <code class="docutils literal notranslate"><span class="pre">command</span></code> to the <a class="reference internal" href="glossary.html#term-DUT"><span class="xref std std-term">DUT</span></a>, unless empty</p></li>
<li><p>if <code class="docutils literal notranslate"><span class="pre">echo:</span> <span class="pre">discard</span></code> is specified, discard next output line (assumed to be
an echo of the command)</p></li>
<li><p>wait for the <code class="docutils literal notranslate"><span class="pre">prompts</span></code>, <code class="docutils literal notranslate"><span class="pre">successes</span></code> or <code class="docutils literal notranslate"><span class="pre">failures</span></code></p></li>
<li><p>if a <code class="docutils literal notranslate"><span class="pre">name</span></code> is defined, log the result for this command (as soon as a prompt or a message is matched)</p></li>
<li><p>if a <code class="docutils literal notranslate"><span class="pre">successes</span></code> or <code class="docutils literal notranslate"><span class="pre">failures</span></code> was matched, wait for the <code class="docutils literal notranslate"><span class="pre">prompts</span></code></p></li>
</ul>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The interactive test action expects the prompt to be already matched
before it starts. If this is not the case, then wait for the prompt by
adding an empty <code class="docutils literal notranslate"><span class="pre">command</span></code> directive as described below.</p>
</div>
<p>A u-boot interactive test might look like:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">    </span><span class="nt">interactive</span><span class="p">:</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">network</span>
<span class="w">      </span><span class="nt">prompts</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">[</span><span class="s">&quot;=&gt;&quot;</span><span class="p p-Indicator">,</span><span class="w"> </span><span class="s">&quot;/</span><span class="nv"> </span><span class="s">#</span><span class="nv"> </span><span class="s">&quot;</span><span class="p p-Indicator">]</span>
<span class="w">      </span><span class="nt">echo</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">discard</span>
<span class="w">      </span><span class="nt">script</span><span class="p">:</span>
<span class="w">      </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">dhcp</span>
<span class="w">        </span><span class="nt">command</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">dhcp</span>
<span class="w">        </span><span class="nt">successes</span><span class="p">:</span>
<span class="w">        </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">message</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;DHCP</span><span class="nv"> </span><span class="s">client</span><span class="nv"> </span><span class="s">bound</span><span class="nv"> </span><span class="s">to</span><span class="nv"> </span><span class="s">address&quot;</span>
<span class="w">        </span><span class="nt">failures</span><span class="p">:</span>
<span class="w">        </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">message</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;TIMEOUT&quot;</span>
<span class="w">          </span><span class="nt">exception</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">InfrastructureError</span>
<span class="w">          </span><span class="nt">error</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;dhcp</span><span class="nv"> </span><span class="s">failed&quot;</span>
<span class="w">      </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">setenv</span>
<span class="w">        </span><span class="nt">command</span><span class="p">:</span><span class="w"> </span><span class="s">&quot;setenv</span><span class="nv"> </span><span class="s">serverip</span><span class="nv"> </span><span class="s">{SERVER_IP}&quot;</span>
<span class="w">      </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">wait for the prompt</span>
<span class="w">        </span><span class="nt">command</span><span class="p">:</span>
</pre></div>
</div>
<section id="id2">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">name</a><a class="headerlink" href="#id2" title="Permalink to this heading">¶</a></h3>
<p>The name of the <a class="reference internal" href="results-intro.html#results-test-suite"><span class="std std-ref">test suite</span></a>.</p>
</section>
<section id="prompts">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">prompts</a><a class="headerlink" href="#prompts" title="Permalink to this heading">¶</a></h3>
<p>The list of possible prompts for the interactive session. In many cases,
there is just one prompt, but if shell has different prompts for different
states, it can be accommodated. (Prompts can also include regexps, as any
other match strings).</p>
</section>
<section id="echo">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">echo</a><a class="headerlink" href="#echo" title="Permalink to this heading">¶</a></h3>
<p>If set to <code class="docutils literal notranslate"><span class="pre">discard</span></code>, after each sent <code class="docutils literal notranslate"><span class="pre">command</span></code> of a <code class="docutils literal notranslate"><span class="pre">script</span></code>, discard
the next output line (assumed to be an echo of the command). This option
should be set when interacting with shell (like u-boot shell) that will echo
the command, to avoid false positive matches. Note that this options applies
to every <code class="docutils literal notranslate"><span class="pre">command</span></code> in the script. If you need different value of this
option for different commands, you would need to group them in different
<code class="docutils literal notranslate"><span class="pre">script</span></code>’s.</p>
</section>
<section id="script">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">script</a><a class="headerlink" href="#script" title="Permalink to this heading">¶</a></h3>
<p>The list of commands to send and what kind of output to expect:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">name</span></code>: If present, log the result (pass/fail) of this command
under the given name (as a testcase). If not present, and the command
fails, the entire test will fail (with <a class="reference internal" href="dispatcher-design.html#test-error-exception"><span class="std std-ref">TestError</span></a>).</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">command</span></code>: The command (string) to send to device, followed by newline.
The command can use variables that will be substituted with live data,
like <code class="docutils literal notranslate"><span class="pre">{SERVER_IP}</span></code>. If value is empty (<code class="docutils literal notranslate"><span class="pre">command:</span></code> in YAML), nothing
is sent, but output matching (prompts/successes/failures) will be
performed as usual. (Note that empty <code class="docutils literal notranslate"><span class="pre">command:</span></code> is different from empty
string <code class="docutils literal notranslate"><span class="pre">command:</span> <span class="pre">&quot;&quot;</span></code>. In the latter case, just a newline will be sent
to device.)</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">failures</span></code> and <code class="docutils literal notranslate"><span class="pre">successes</span></code>: Each optional. If present, check the
device output for the given patterns.</p></li>
</ul>
<p><code class="docutils literal notranslate"><span class="pre">successes</span></code> should be a list of dictionaries with just one key:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">message</span></code>: The string (or regexp) to match. Substring match is
performed, so care should be taken to reliably encode the match pattern.
(E.g. <code class="docutils literal notranslate"><span class="pre">message:</span> <span class="pre">4</span></code> would match “4” appearing anywhere in the output,
e.g. “14” or “41”).</p></li>
</ul>
<p><code class="docutils literal notranslate"><span class="pre">failures</span></code> should be a list of dictionaries with:</p>
<ul class="simple">
<li><p><code class="docutils literal notranslate"><span class="pre">message</span></code>: The string (or regexp) to match. Substring match is performed.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">exception</span></code> (optional): If the message indicates a fatal problem,
an exception can be raised, one of:
<a class="reference internal" href="dispatcher-design.html#infrastructure-error-exception"><span class="std std-ref">InfrastructureError</span></a>,
<a class="reference internal" href="dispatcher-design.html#job-error-exception"><span class="std std-ref">JobError</span></a>,
<a class="reference internal" href="dispatcher-design.html#test-error-exception"><span class="std std-ref">TestError</span></a>. If not present, the error
is not fatal and will be recorded just as a failed testcase in test
results. (If this is a named command; as mentioned above, failure of
unnamed (“not a testcase”) command leads to implicit TestError).</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">error</span></code>: if defined, the exception message which will appear in the job log</p></li>
</ul>
<p>If <code class="docutils literal notranslate"><span class="pre">successes</span></code> is defined, but LAVA matches one of the prompts
instead, an error will be recorded (following the logic that the lack
of expected success output is an error). This means that in many cases
you don’t need to specify <code class="docutils literal notranslate"><span class="pre">failures</span></code> - any output but the successes
will be recorded as an error.</p>
<p>However, if <code class="docutils literal notranslate"><span class="pre">successes</span></code> is not defined, then matching a prompt will
generate a passing result (this is useful for interactive commands
which don’t generate any output on success; of course, in this case
you would need to specify <code class="docutils literal notranslate"><span class="pre">failures</span></code> to catch them).</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference internal" href="interactive.html#writing-tests-interactive"><span class="std std-ref">Writing an interactive test action</span></a></p>
</div>
</section>
</section>
<section id="monitors">
<span id="monitor-test-action"></span><span id="index-2"></span><h2><a class="toc-backref" href="#contents" role="doc-backlink">Monitors</a><a class="headerlink" href="#monitors" title="Permalink to this heading">¶</a></h2>
<p>Test jobs using Monitors <strong>must</strong>:</p>
<ol class="arabic simple">
<li><p>Be carefully designed to automatically execute after boot.</p></li>
<li><p>Emit a unique <code class="docutils literal notranslate"><span class="pre">start</span></code> string:</p>
<ol class="arabic simple">
<li><p>Only once per boot operation.</p></li>
<li><p>Before any test operation starts.</p></li>
</ol>
</li>
<li><p>Emit a unique <code class="docutils literal notranslate"><span class="pre">end</span></code> string:</p>
<ol class="arabic simple">
<li><p>Only once per boot operation.</p></li>
<li><p>After all test operations have completed.</p></li>
</ol>
</li>
<li><p>Provide a regular expression which matches all expected test output
and maps the output to results <strong>without</strong> leading to excessively
long test case names.</p></li>
</ol>
<p><code class="docutils literal notranslate"><span class="pre">start</span></code> and <code class="docutils literal notranslate"><span class="pre">end</span></code> strings will match part of a line but make sure
that each string is long enough that it can only match once per boot.</p>
<p>If <code class="docutils literal notranslate"><span class="pre">start</span></code> does not match, the job will timeout with no results.</p>
<p>If <code class="docutils literal notranslate"><span class="pre">end</span></code> does not match, the job will timeout but the results (of
the current boot) will already have been reported.</p>
<section id="id3">
<h3><a class="toc-backref" href="#contents" role="doc-backlink">name</a><a class="headerlink" href="#id3" title="Permalink to this heading">¶</a></h3>
<p>The name of the <a class="reference internal" href="results-intro.html#results-test-suite"><span class="std std-ref">test suite</span></a>.</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">test</span><span class="p">:</span>
<span class="w">    </span><span class="nt">monitors</span><span class="p">:</span>
<span class="w">    </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">tests</span>
<span class="w">      </span><span class="nt">start</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">BOOTING ZEPHYR</span>
<span class="w">      </span><span class="nt">end</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">PROJECT EXECUTION SUCCESSFUL</span>
<span class="w">      </span><span class="nt">pattern</span><span class="p">:</span><span class="w"> </span><span class="s">&#39;(?P&lt;test_case_id&gt;\d+</span><span class="nv"> </span><span class="s">*-</span><span class="nv"> </span><span class="s">[^-]+)</span><span class="nv"> </span><span class="s">(?P&lt;measurement&gt;\d+)</span><span class="nv"> </span><span class="s">tcs</span><span class="nv"> </span><span class="s">=</span><span class="nv"> </span><span class="s">[0-9]+</span><span class="nv"> </span><span class="s">nsec&#39;</span>
<span class="w">      </span><span class="nt">fixupdict</span><span class="p">:</span>
<span class="w">        </span><span class="nt">PASS</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">pass</span>
<span class="w">        </span><span class="nt">FAIL</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">fail</span>
</pre></div>
</div>
<p>If the device output is of the form:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>***** BOOTING ZEPHYR OS v1.7.99 - BUILD: Apr 18 2018 10:00:55 *****
|-----------------------------------------------------------------------------|
|                            Latency Benchmark                                |
|-----------------------------------------------------------------------------|
|  tcs = timer clock cycles: 1 tcs is 12 nsec                                 |
|-----------------------------------------------------------------------------|
| 1 - Measure time to switch from ISR back to interrupted thread              |
| switching time is 107 tcs = 1337 nsec                                       |
|-----------------------------------------------------------------------------|

...

PROJECT EXECUTION SUCCESSFUL
</pre></div>
</div>
<p>The above regular expression can result in test case names like:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>1_measure_time_to_switch_from_isr_back_to_interrupted_thread_switching_time_is
</pre></div>
</div>
<p>The raw data will be logged as:</p>
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>test_case_id: 1 - Measure time to switch from ISR back to interrupted thread              |
| switching time is
</pre></div>
</div>
<div class="admonition caution">
<p class="admonition-title">Caution</p>
<p>Notice how the regular expression has not closed the match
at the end of the “line” but has continued on to the first
non-matching character. The test case name then concatenates all
whitespace and invalid characters to a single underscore. LAVA uses pexpect
to perform output parsing. pexpect docs explain how to find line ending
strings: <a class="reference external" href="https://pexpect.readthedocs.io/en/stable/overview.html#find-the-end-of-line-cr-lf-conventions">https://pexpect.readthedocs.io/en/stable/overview.html#find-the-end-of-line-cr-lf-conventions</a></p>
</div>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="sa">r</span><span class="s1">&#39;(?P&lt;test_case_id&gt;\d+ *- [^-]+) (?P&lt;measurement&gt;\d+) tcs = [0-9]+ nsec&#39;</span>
</pre></div>
</div>
<p>The test_case_id will be formed from the match of the expression <code class="docutils literal notranslate"><span class="pre">\d+</span>
<span class="pre">*-</span> <span class="pre">[^-]+</span></code> followed by a single space - but <strong>only</strong> if the rest of the
expression matches as well.</p>
<p>The measurement will be taken from the match of the expression <code class="docutils literal notranslate"><span class="pre">\d+</span></code>
preceded by a single space and followed by the <strong>exact</strong> string <code class="docutils literal notranslate"><span class="pre">tcs</span> <span class="pre">=</span>
<span class="pre">``</span> <span class="pre">which</span> <span class="pre">itself</span> <span class="pre">must</span> <span class="pre">be</span> <span class="pre">followed</span> <span class="pre">by</span> <span class="pre">a</span> <span class="pre">number</span> <span class="pre">of</span> <span class="pre">digits,</span> <span class="pre">then</span> <span class="pre">a</span> <span class="pre">single</span>
<span class="pre">space</span> <span class="pre">and</span> <span class="pre">finally</span> <span class="pre">the</span> <span class="pre">**exact**</span> <span class="pre">string</span> <span class="pre">``nsec</span></code> - but only if the rest
of the expression also matches.</p>
<div class="admonition seealso">
<p class="admonition-title">See also</p>
<p><a class="reference external" href="https://docs.python.org/3/howto/regex.html">Regular Expression HOWTO for Python3</a></p>
</div>
</section>
</section>
</section>


    </div>
      
  </div>
</div>
<footer class="footer">
  <div class="container">
    <p class="pull-right">
      <a href="#">Back to top</a>
      
    </p>
    <p>
        &copy; Copyright 2010-2019, Linaro Limited.<br/>
      Created using <a href="http://sphinx-doc.org/">Sphinx</a> 5.3.0.<br/>
    </p>
  </div>
</footer>
  </body>
</html>